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A RELIABILITY  TEST  METHOD  FOR  “ONE-SHOT"  ITEMS 


H.  J.  Langlie 

Aeronutronic  Division,  Ford  Motor  Company 

1.  INTRODUC TION.  As  a result  of  many  reliability  problems  which  have 
plagued  procuring  agencies  in  the  missile  and  space  programs,  increased 
emphasis  has  been  placed  on  the  development  of  improved  reliability  demon- 
stration test  methods.  In  this  connection,  the  Army  has  advocated  increased 
use  of  the  "test-to-failure“  concept  to  establish  the  existence  of  satisfactory 
margins  of  reliability  with  respect  to  critical  factors.  This  philosophy  has 
the  advantage  in  that  statistical  statements  can  be  made  regarding  reliability 
on  the  basis  of  relatively  small  samples.  This  paper  will  discuss  the  appli- 
cation of  this  general  concept  to  a particular  class  of  hardware. 

In  essence,  the  "test-to-failure"  concept  involves  submitting  a test 
specimen  to  an  increasing  environment  or  load  stress  until  failure  is  detected. 
By  observing  the  statistical  behavior  of  the  stresses  at  which  failure  occurs, 
the  lower  limit  of  stress  below  which  the  probability  of  failure  is  very  small 
can  be  selected  by  using  the  mean  and  standard  deviation  of  these  data. 

Robert  Lusser  (reference  1)  advocated  the  safety  margin  concept  in  inter- 
preting these  data.  As  is  shown  in  Figure  1,  the  larger  the  value  of  k 
(which  is  the  distance  from  the  mean  strength  to  the  upper  limit  of  the  oper- 
ating applied  stress  divided  by  the  standard  deviation  of  the  data),  the 
greater  the  reliability  of  the  specimen  with  respect  to  the  stress  involved. 

In  establishing  the  reliability  objectives  for  the  Shillelagh  Program,  the 
Army  Missile  Command  required  that  safety  margins  be  demonstrated  in 
the  test  laboratory  for  Shillelagh  components  with  respect  to  critical  environ- 
mental stress  factors.  Many  of  these  components  are  of  a “go-no-go"  type 
such  as  thermal  batteries,  electrical  relays,  and  other  short-lived  equip- 
ment items.  In  most  cases,  little  information  was  available  prior  to  test 
regarding  the  nature  of  the  standard  deviation  of  the  distribution  of  strenghts 
for  these  parts.  Furthermore,  it  was  desired  to  perform  a laboratory  test 
involving  a minimum  number  of  samples.  A review  of  attribute  sensitivity 
testing  techniques  such  as  the  Up  and  Down  method  (reference  2)  and  the 
Probit  method  (reference  3)  indicated  that  these  methods  cannot  be  applied 
satisfactorily  under  the  sample  size  and  technical  limitations  imposed.  As 
a result,  a study  was  made  to  develop  a method  for  selecting  stress  levels 
for  testing  which  required  no  a'priori  assumption  regarding  the  standard 
deviation  of  the  unknown  strength  distribution  and  co\ild  be  performed  satis- 
factorily with  sample  sizes  of  the  order  of  fifteen  or  twenty. 


FIGORE  1 
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Detailed  empirical  investigations  were  performed,  using  Monte  Carlo, 
methods  with  a high  speed  digital  computer,  to  develop  a satisfactory- 
algorithm  for  determining  successive  stress  test  levels  as  the  experiment 
proceeds.  Exact  maximum  likelihood  equations  were  used  to  calculate  the 
statistical  estimates,  /4  , and  c , of  the  population  parameters,  W- 
and  O',  of  the  distribution  of  part  strengths.  By  repeated  simulations  of 
experiments  for  sampler  sizes  ranging  from  ten  to  900,  empirical  curves 
were  obtained  showing  the  variance  in  the  calculated  estimates  vs  sample 
size.  After  this  study  was  complete,  a technical  report  was  prepared 
(reference  4).  The  paper  today  will  discuss  some  significant  results  con- 
tained in  this  report. 

Since  the  completion  of  the  study,  several  applications  have  been  made 
in  reliability  testing  electrical  and  mechanical  components.  One  such 
application  is  presented  in  this  paper  by  way  of  illustration. 

2.  DISC U SSION . In  order  to  proceed  with  the  discussion  of  the  test 
method,  a specific  definition  is  given  regarding  the  terms  "stress”  and 
"strength"  as  follows; 

Stress  is  a test  factor,  such  as  environmental  level  or  force  level 
which  is  applied  to  the  test  specimen.  Operational  stresses  repre- 
sent the  mix  of  en-vironmental  or  load  conditions  that  can  be  expec- 
ted to  be  imposed  on  a typical  specimen  during  its  life.  During  the 
conduct  of  a "one-shot"  test,  the  stress  represents  the  applied  test 
factor  which  is  varied  in  magnitude  from  specimen  to  specimen  in 
a systematic  manner. 


Strength  is  a property  ascribed  to  a specimen  such  that  if  the  stress 
imposed  on  the  part  is  greater  than  the  strength  of  the  part,  the 
part  will  fail.  Conversely,  if  the  stress  is  less  than  the  strength  of 
the  part,  the  part  will  not  fail. 

Failure  in  the  sense  used  above  is  a general  term  referring  to  unsat- 
isfactory completion  of  fxinction,  out  of  tolerance  performance,  breakage, 
or  other  e-vidence  of  malfunction.  For  each  test  attempt  wherein  a stress 
is  applied  to  a specimen,  there  is  associated  an  outcome  which  is  a binary 
variable;  success  or  failure. 

It  is  assumed  that,  given  a homogenous  sample  of  replicate  specimens, 
the  part  strengths  are  distributed  normally  with  an  unknown  mean  and 
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standard  deviation.  The  purpose  of  the  test  method  is  to  select  stress 
levels  in  such  a way  as  to  generate  outcomes  which  can  then  be  used  to 
calculate  statistical  estimates  of  the  parameters  of  the  distribution  of 
part  strengths. 

3.  PERFORMING  THE  TEST.  In  undertaking  to  perform  the  "one- 
shot”  test,  there  are  three  steps  to  be  taken  which  should  be  followed 
regardless  of  the  nature  of  the  application.  They  are: 

1.  Establish  the  criteria  of  failure,  or  acceptance. 

2.  Determine  the  test  interval. 

3.  Select  the  stress  levels. 

The  last  of  these  three  steps,  selecting  the  stress  levels,  proceeds  con- 
current with  the  actual  performance  of  the  test.  These  three  steps  are 
discussed  below. 

3.1  Establishing  Failure  Criteria, 

It  is  very  important  that  painstaking  care  be  given  to  the  set  of  ground 
rules  which  will  be  followed  for  differentiating  between  a success  or  a 
failure.  For  purposes  of  reliability  testing,  this  means  that  a careful 
enumeration  should  be  made  of  all  undesirable  responses  of  a test  speci- 
men, such  as  particular  modes  of  failure,  out  of  tolerance  performance, 
and  any  other  mode  of  unacceptable  product  performance.  These  criteria 
are  the  very  basis  for  establishing  product  assurance  in  the  laboratory  and 
should  be  reviewed  and  approved  by  all  qualified  parties  having  a technical 
interest  in  the  product. 


3.  2 Determining  the  Test  Interval. 

In  order  to  proceed  with  a generation  of  stress  levels  for  testing  pur- 
poses, it  is  necessary  to  choose  a test  interval  which  is  used  as  a basis 
for  the  stress  sequencing  method.  This  interval  should  be  selected  large 
enough  to  include  all  possible  ranges  of  strengths  of  the  parts  to  be  tested. 
This  interval  can  be  made  conservatively  large,  since  the  "one-shot"  method 
has  been  designed  to  cause  the  stress  levels  to  be  generated  in  the  vicinity 
of  interest  (i.  e.  , in  the  vicinity  of  the  distribution  of  strengths)  as  the 
test  proceeds.  As  a sample  illustration,  the  range  for  a drop  height  test 
for  glass  containers  designed  to  withstand  say,  a six  inch  drop,  could  be 
chosen  to  have  a lower  limit  of  zero  and  an  upper  limit  of  three  feet.  The 
method  of  analysis  of  the  data  is  such  that  the  particular  choice  of  the 
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endpoints  of  the  test  interval  do  not  have  an  appreciable  effect  on  the 
results  for  sample  sizes  of  fifteen  or  more.  In  the  event  that  the  test 
interval  turns  out,  as  the  test  proceeds,  to  be  inappropriately  chosen, 
then  the  stress  levels  will  tend  to  converge  towards  one  limit  or  the 
other.  In  such  an  event,  particularly  in  the  case  of  reliability  testing, 
convergence  towards  the  lower  level  is  usually  indicative  of  a totally 
unsatisfactory  product,  whereas  convergence  towards  the  upper  limit 
can  be  shown  to  be  statistically  acceptable  by  use  of  the  likelihood  ratio 
test. 


In  Figure  2 there  is  represented  the  results  of  an  actual  "one-shot" 
test  on  thermal  batteries  to  determine  the  reliability  with  regard  to  high 
temperature.  In  this  instance,  the  batteries  were  designed  to  perform 
reliably  at  145  F.  On  the  basis  of  conservative  engineering  judgement 
and  some  limited  development  test  data,  the  lower  limit  was  selected 
to  be  100°F  (the  level  at  which  all  thermal  batteries  would  be  expected 
to  perform  satisfactorily)  and  the  higher  limit  was  selected  to  be  350‘^F 
(the  level  at  which  all  thermal  batteries  wovild  be  expected  to  fail). 


3.  3 Selecting  the  Stress  Levels. 

Once  the  test  interval  and  failure  criteria  have  been  established,  the 
test  commences  by  selecting  the  first  stress  level  at  the  midpoint  of  the 
interval.  After  exposing  the  first  specimen  to  this  environmental  level 
and  activating  it,  a one  or  zero  is  recorded  to  indicate  the  outcome  as  a 
success  or  failure  respectively  (see  Figure  2). 


The  general  riile  for  obtaining  the  (n  + 1) 


st 


stress  level,  having  corn- 
starting  at 


plete^  n trials,  is  to  work  backward  in  the  test^^sequence, 
the  n trial,  until  a previous  trial  (call  is  the  p trisi^)  is  found  sucij. 


that  there  are  as  many  successes  as  failures  in  the  p 
trials.  The  (n  + 1)^ 


th'. 


th 


through  the 

s|gess  level  is  then  obtained  by  averaging  the  n 
stress  level  with  the  p stress  level.  If  there  exists  no  previous  stress 
level  satisfying  the  requirement  sta^d  above,  then  the  (n  + 1)  stress 
level  is  obtained  by  averaging  the  n stress  level  with  the  lower  or  upper 


stress  limits  of  the  test  interval  according  to  whether  the  n 
a failure  or  a success. 


th 


result  was 


To  illustrate,  suppose  it  is  desired  to  find  the  second  stress  level  in 
Figure  2.  Since  there  was  only  one  previous  observation  (i.  e.  , first 
unit  failed)  it  is  not  possible  to  find  a stress  level  where  all  intervening 
results  even  out.  That  is,  the  second  stress  level  is  obtained  by  averaging 
the  first  with  the  lower  limit.  To  find  the  eighth  stress  level,  it  is  ob- 
served that  results  from  test  4 through  7 (i.  e.  , the  last  four  results)  can- 
cel each  other  out.  Thus,  the  eighth  stress  level  is  obtained  by  averaging 
the  fourth. 


^op£-eporn 
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As  a final  example,  it  is  observed  that  after  the  twelfth  test  has  been 
completed,  there  again  exists  no  previous  stress  level  for  which  the 
number  of  failures  equals  the  number  of  successes.  Since  the  twelfth 
test  was  a failure,  the  thirteenth  stress  level  is  obtained  by  averaging  the 
twelfth  stress  level  with  the  lower  limit. 

As  an  aid  in  identifying  the  important  parameters  of  the  test,  the  stress 
level  is  designated  by  the  letter  s and  the  outcome  is  designated  by  the 
letter  u.  The  lower  limit  of  the  test  interval  is  designated  A and  the 
upper  limit  is  designated  B.  Upon  the  conclusion i of  the  test,  the  stress 
values,  (s  s , , s ),  and  the  corresponding  outcomes,  (u  , u , . , . ,u  ), 

JL  w IN  X i.N 

where  N equals  the  test  sample  size,  are  used  to  perform  a complete  anal- 
ysis for  hardware  reliability. 

4.  DERIVATION  OF  MAXIMUM  LIKELIHOOD  EQUATIONS.  Consider 
the  random  sample  X = (x^,  x^,  . . . , x^^)  of  N observations  where  the  x^ 

are  independent  random  variable  from  a Gaussian  distribution,  g(x;jU,  o'), 

with  mean  jXand  standard  deviation  a.  Consider  also  an  N-dimensional 

vector  S = (s, , s.,,  . . . s..)  where  A<.s.<B.  From  this,  construct  a third 
'12  N — 1— 

vector  U = (u^^,  u^,  ....  vl^)  where 

u.  = 1 if  s.<  X. 

1 11 

u.  = O if  s.  > X. 

1 1 1 

th 

The  variable  x.  is  called  the  strength  of  the  i part;  s.  is  called  the 

applied  stress  level  for  the  i part,  and  u.  is  called  the  outcome  of  the 
th  ^ 

•'test"  on  the  i part.  The  outcome  u = 1 is  called  a success  (i.  e.  , the 
applied  stress  was  less  than  the  part  strength)  and,  conversely,  u = O is 
called  a failure. 

The  object  of  this  development  is  to  obtain  formulas  for  calculating  the 

estimates  LL  and  tr  of  u and  cr  , given  only  S and  U.  This  will  be 
' e e ' 

accomplished  by  obtaining  values  of  /x.  and  cr  which  maximize  the  like- 
lihood  (i.  e.  , probability)  of  obtaining  §ie  outcome  U given  S. 

The  probability  of  outcome  u^  given  s.  can  be  written 
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(1)  p.  = Prob 


^ O i 

[uil  sj  = ujg  g{v;yU.,  0-)dv  + (l  - u.^_^g(v;^,  cr)d 


The  probability  of  outcome  U can  be  written  as  the  product  of  the  proba- 
bilities of  the  individual  u^  since  the  are  independent. 

(2)  p[u]  = ^ p.  r) 

The  expression  pfu].  when  regarded  as  a function  of  the  population  parame. 
ters/X.and  (r,  becomes  the  likelihood  function  for  outcome  U (ref.  5), 

To  find  the  values  of  //and  cr  (now  regarded  as  variables)  which  maxi- 
mize L,  we  differentiate  (2)  with  respect  to //-and  cr  and  solve  the  system 

(3)  31nL/8/t=0 

81n  L/8//.  = O 

where  the  logarithm  of  L is  used  to  simplify  the  algebra. 

‘"I/2  2 

Letting  t.  = (s^  -//-)/(r,  g^(v)  = (2tr)  ” ' exp  (-v  /2),  the  normalized 
Gaussian,  and  remembering  that  u^  can  take  on  only  values  of  O or  1, 
equation  (1)  can  be  re-written  ^ 

(4)  In  p.  = u.  ln[J^g^  (v)dv]  + (1  - upin[f  ^^^(v)dv] 

i J-00 

The  following  definitions  (=)  and  derivations  will  be  helpf\al: 


(5) 

=f^  g (v)  dv 

(6) 

J-00 

dg^(t)/dt  = -tg^(t) 

(7) 

8t/ 8//,  = -l/a 

(8) 

8t/8cr  = -t/c 

Since  InL  = E In  p., 
1 

equations  (3)  become 

(9) 

dlnL/ djU,  = E 81n 

(0-7  O) 
(o->0) 


a?nL/8cr  = S 81n  p./8tr  = O 
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(lO) 

ain  pVaix  = ^ 

•g  (t.) 
i®o' 

8t. 

— + 

(l-upg^(tp 

8t. 

1 

- G(tp 

dfl. 

G(tp 

QfJL 

u. 

1 

1-u. 

1 

(T 

1-G. 
_ 1 

G. 

1 _ 

where  the 

arguments  (and  subscript  "o”)  have  been  omitted  to  simplify 

writing. 

Similarly, 

(11) 

81n  p./ 8o-  = ^i^i 

u. 

1 

1-u. 

1 

tr 

1-G. 
L 1 

G. 

1 

- 

Denoting  by  h^  the  expression  in 

brackets 

and  eliminating  the  constant  cr, 

equations  (9)  become 


N 


ainL/a^  = p(^,  <r)  = g.h. 


= O 


N 


ainL/8  O'  = q(^,  (t)  = S t.g.h.  = O 

1=1 


Equations  (12)  are  valid  only  if  the  value  of  cr  which  satisfies  the  maxi- 
mum likelihood  equations  is  non- zero.  A quick  examination  of  the  data 
can  be  made  to  determine  if  a non-zero  o'  is  a maximum  likelihood  solu- 
tion. If  the  maximum  stress  level  at  which  a success  occurred  is  greater 
than  the  minimum  level  at  which  a failure  occurred,  than  a non- zero  cr 
satisfies  equations  (12).  If  this  statement  is  not  true,  then  cr  = O represents 
a maximum  likelihood  estimate  for  the  standard  deviation  and  the  maximum 
likelihood  estimate  of  the  mean  is  a connected  interval  contained  between 
the  maximum  failure  stress  level  and  the  minimum  success  stress  level 
which  represent  the  lower  and  upper  bounds  of  the  interval  respectively. 

The  latter  situation  illustrates  an  outcome  which  in  fact  must  be  achieved 
if  all  of  the  part  strengths  were  concentrated  at  a mass  point  within  the 
above  mentioned  interval.  The  maximum  likelihood  corresponding  to 
this  outcome  is  one.  Although  unique  estimates  cannPt  be  obtained  for 
U and  a in  such  an  instance,  it  is  possible  to  provide  a basis  for  deci- 
sion-making,  using  the  likelihood  ratio  statistic  along  with  a suitably 
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constructed  hypothesis  (reference  4).  A result  of  this  type  is  referred 
to  as  degenerate.  It  should  be  mentioned  that,  for  a fixed  popvilation 
standard  deviation, ^ 0,  the  probability  of  obtaining  a degenerate  result 
approaches  0 as  the  sample  size  becomes  large.  The  proof  of  these 
statements  is  contained  in  reference  4 and  is  beyond  the  scope  of  this 
paper. 


5.  STATISTICAL  PROPERTIES  OF  THE  MAXIMUM  LIKELIHOOD 
ESTIMATES.  The  preceding  sections  describe  the  method  for  selecting 
stress  levels  and  the  likelihood  equations  for  calculating and  O'  . 

In  order  to  make  practical  applications  of  the  method,  however,  infor- 
mation is  required  regarding  the  statistical  properties  of  the  maximum 
likelihood  estimates  corresponding  to  various  sample  sizes.  To  facili- 
tate obtaining  this  information,  an  extensive  computer  simulation  was 
carried  out  using  Monte  Carlo  techniques.  In  this  way,  hundreds  of  values 
of  M and  O'  could  be  obtained  corresponding  to  himdreds  of  simulated 
experiments  using  random  numbers  for  part  strengths.  Statistical 
summaries  were  then  obtained  and  the  variance  of  the  estimates  of  the 
parameters  empirically  derived. 

To  perform  the  simulation,  a standard  interval,  A = 1,  B = 1,  was 
chosen.  The  sampling  of  strengths  was  simxilated  by  converting  the  sum 
of  twelve  two-digit  numbers,  constructed  from  a file  of  one  millions  ran- 
dom digits,  to  a random  deviate  with  popxdation  mean/^and  standard 
deviation.  O'  . Two  poptilations  were  employed:  f~*-  = 0 and  O' - 0..25, 
and/4.  = 0.  2 and  O'  = 0.  1.  For  each  pop\ilation,  one  hundred  runs,  each 
consisting  of  N samples,  were  made  for  N = 4 through  15,  20,  25,  30 
and  35,  with  an  additional  four  hundred  runs  for  N = 15  and  N = 30  for 
additional  information  on  the  distributions  of  the  estimates.  Finally, 
four  runs  of  N - 900  were  made  for  each  population  to  empirically  inves- 
tigate the  asymptotic  convergence  of  (/4  , cT'  ) to  (/^,  <7'). 

e e 

For  each  set  of  100  runs  at  a fixed  sample  size,  the  mean  and  variance 
were  averaged  separately  and  plotted  as  shown  in  Figures  3 and  4.  Straight 
lines  were  then  fitted  to  the  data,  recognizing  that  some  spurious  effects 
are  introduced  for  small  sample  sizes  (i.  e. , 15  or  less)  due  to  the  dis- 
creteness of  the  admissible  outcomes  which  are  possible.  (For  sample 
size  of  N,  only  2^  outcomes  are  possible  corresponding  to  the  2^  possi- 
ble configurations  of  0 and  1. ) 

Based  on  the  resialts  of  Figures  3 and  4,  the  variance  of  the  estimates 
are  given  by: 


FIGURE  3 


FIGURE  4. 
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V(^^)  = 2.  5(t^/N 

V(o-  ) = 3.2o-Vn 
e 


In  order  to  use  these  formulas  in  practical  applications,  the  unbiased 
estimate  of  a,  denoted  o',  is  substituted  for  cr  in  the  above  formulas,  '.  The 
relationship  between  the  unbiased  estimate  and  the  maximum  likelihood 
estimate  is  given  empirically  in  Figure  5.  The  variance  of  the  unbiased 
estimate  can  be  calculated  using  the  relation 


3.  2a-^ 
N(3^ 


3.  2cr' 


Np 


The  above  formula  is  sufficiently  accurate  for  sample  sizes  on  the  order  of 
50  or  greater,  wherein  the  distribution  of  the  estimate  of  the  standard  devia 
tion  approaches  the  normal  distribution.  For  smaller  size  samples  it  was 
observed  from  the  empirical  study  that  na/c  approximately  follows  the  chi- 
square  distribution  with  n degress  of  freedom  where  n is  given  by 

2 

n = O.  625p  N (reference  4) 
where  N is  the  sample  size  and  (3  is  given  by  Figure  5. 
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